Medical Image Retrieval based on Combination of Visual Semantic and Local Features

Content-based medical image retrieval is an important tool for doctors in their daily activity. In this paper, we propose a novel image retrieval framework to combine visual concept and local features. To obtain visual semantic representation of the image, we first construct a graph model by feature distance and density similarity, and then a graph-based semi-supervised learning method is applied to get the membership degree of query images. Meanwhile, the dense SIFT feature of the image patches is extracted and described by bag of visual words as local features. Besides, we design a similarity measurement based on visual concept and local feature rather than using low level features only. We evaluate the proposed algorithm in ImageCLEFmed dataset. The results demonstrate that our method represents the visual semantic of images effectively, and compares favorably to state-of-the-art approaches based on single low level features in retrieval performance.